Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](planner) should always execute projection plan #17885

Merged
merged 1 commit into from
Mar 22, 2023

Conversation

morrySnow
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Problem summary

  1. should always execute projection plan, whatever the statement it is.
  2. should always execute projection plan, since we only have vectorized engine now

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Mar 17, 2023
@morrySnow morrySnow added usercase Important user case type label dev/1.2.3 labels Mar 17, 2023
@morrySnow morrySnow force-pushed the fix_insert_projection branch from 257e5a9 to 8ae8408 Compare March 19, 2023 13:05
@morrySnow
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Mar 19, 2023

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.6 seconds
stream load tsv: 465 seconds loaded 74807831229 Bytes, about 153 MB/s
stream load json: 25 seconds loaded 2358488459 Bytes, about 89 MB/s
stream load orc: 74 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230321122214_clickbench_pr_117976.html

@morrySnow morrySnow force-pushed the fix_insert_projection branch from 8ae8408 to ac16382 Compare March 21, 2023 05:28
@morrySnow
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 22, 2023
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit a4b151e into apache:master Mar 22, 2023
@morrySnow morrySnow deleted the fix_insert_projection branch March 22, 2023 02:53
morningman pushed a commit that referenced this pull request Mar 28, 2023
1. should always execute projection plan, whatever the statement it is.
2. should always execute projection plan, since we only have vectorized engine now
gnehil pushed a commit to gnehil/doris that referenced this pull request Apr 21, 2023
1. should always execute projection plan, whatever the statement it is.
2. should always execute projection plan, since we only have vectorized engine now
mongo360 pushed a commit to mongo360/doris that referenced this pull request Jul 12, 2023
1. should always execute projection plan, whatever the statement it is.
2. should always execute projection plan, since we only have vectorized engine now
SWJTU-ZhangLei pushed a commit to SWJTU-ZhangLei/incubator-doris that referenced this pull request Jul 25, 2023
commit f5d6201
Author: morrySnow <[email protected]>
Date:   Wed Mar 22 10:53:15 2023 +0800

    [fix](planner) should always execute projection plan (apache#17885)

    1. should always execute projection plan, whatever the statement it is.
    2. should always execute projection plan, since we only have vectorized engine now

commit ad61c84
Author: mch_ucchi <[email protected]>
Date:   Mon Mar 27 17:50:52 2023 +0800

    [fix](planner) fix conjunct planned on exchange node (apache#18042)

    sql like:
    select k5, k6, SUM(k3) AS k3
    from (
        select
            k5,
            date_format(k6, '%Y-%m-%d') as k6,
            count(distinct k3) as k3
        from t
        group by k5, k6
    ) AS temp where 1=1
    group by k5, k6;

    will throw exception since conjuncts planned on exchange node, because exchange node cannot handle conjuncts, now we skip exchange node when planning conjuncts, which fixes the bug.
    notice: the bug occurs iff the conjunct is always true like 1=1 above.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. area/planner Issues or PRs related to the query planner dev/1.2.4-merged reviewed usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants